The Tangible Virtual Mirror: New Visualization Paradigm for Navigated Surgery

Real-time in-situ visualization of virtual data overlaid on its real counterpart is getting increasing attention. Augmented Reality (AR) for intra-operative visualization and navigation has been a subject of intensive research and development during the last decade [2], [9], [12]. Besides accuracy and speed of the system one of the challenges to improve acceptance of medical AR systems is the lack of appropriate visualization paradigms that exploit the potential of these new 3D visualization technologies. In-situ visualization offers a higher degree of freedom for the programmer than classical visualization on a screen. The existing paradigms for interaction with 3D objects are however not satisfactory since the majority of them rotate and move the object of interest. The classic manipulation of virtual objects cannot be used while keeping real and virtual spaces in alignment. Other classic medical imaging user interfaces provide multiple orthographic views of virtual objects, which do not reveal spatial relationships if points of interest are not by accident in the same slice. This paper introduces a simple and efficient interaction paradigm allowing the users to interact with 3D objects and visualize them from arbitrary viewpoints without disturbing the in-situ visualization, or requiring the user to change the viewpoint. We present a virtual, tangible mirror as a new paradigm for interaction with 3D models. The concept borrows its visualization paradigm in some sense from methodology used by dentists to examine the oral cavity without constantly changing their own viewpoint or moving the patients head. The paper also presents a medical application for the hand-held virtual mirror in order to demonstrate its crucial role in intra-operative AR visualization for computer aided spine surgery. Results not only include simulated optical mirrors but also introduce the concept of medical imaging mirrors reflecting volume rendering. This new concept can be easily combined with all kinds of visualization techniques, e.g. digitally reconstructed radiographs (DRR), which is highly interesting since mirroring X-rays is not physically possible, but could be perceptually accepted and extremely useful for 3D medical in-situ visualization. CR Categories: I.3.6 [Methodology and Techniques]: Interaction ∗e-mail: [email protected] †e-mail: [email protected] ‡e-mail: [email protected] techniques—;